Multi-Level Cross-Modal Semantic Alignment Network for Video–Text Retrieval

This paper strives to improve the performance of video–text retrieval. To date, many algorithms have been proposed facilitate similarity measure retrieval from single global semantic multi-level semantics. However, these methods may suffer following limitations: (1) largely ignore relationship which results in levels are insufficient; (2) it is incomplete constrain real-valued features different modalities be same space only through feature distance measurement; (3) fail handle problem that distributions attribute labels heavily imbalanced. overcome above limitations, this proposes a novel cross-modal alignment network (MCSAN) for by jointly modeling on global, entity, action and unified deep model. Specifically, both video text first decomposed into carefully designing spatial–temporal learning structures. Then, we utilize KLDivLoss parameter-share projection layer as statistical constraints ensure representations projected common space. In addition, focal binary cross-entropy (FBCE) loss function presented, effort model unbalanced distribution MCSAN practically effective take advantage complementary information among four levels. Extensive experiments two challenging datasets, namely, MSR-VTT VATEX, show viability our method.